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A STUDY OF THE LONG-TERM EFFECTIVENESS 
OF THE READING RECOVERY PROGRAM 

REPORT SUMMARY 

Authors: Jan Donley and Nancy Baenen 



BACKGROUND 

The Wake County Public School System (WCPSS) first implemented Reading Recovery in 
1990-91 and expanded the program in ;991~92. The goal of this nationally recognised 
program is to provide intensive help to first graders who are having difficulty learning to 
read before problems become too severe. It is hoped that this preventative approach will 
eliminate, or substantially reduce, the need for future remedial help. 

Reading Recovery staff collected data with the Clay Diagnostic Survey during the year 
students were involved. The Chapter 1 Evaluation Specialist collected follow-up data for the 
students served in 1990-91 and 1991-92 to measure their success after leaving the program. 
This included analyses of retention, special education, and regular Chapter 1 program 
participation rates as well as surveys of second grade teachers 1 perceptions of 1990-91 
students regarding their reading performance (as of spring, 1992). All analyses included 
comparison groups of similar students. 



MAJOR FINDINGS 

1. Annual site reports by Reading Recovery staff found positive short-term results for both 
cohorts of students served. 

• Seventy-seven percent and 73% of the 1990-91 and 1991-92 Reading Recovery 
students who completed the program, respectively, were able to be discontinued 
successfully from the program. 

• Reading Recovery students with a complete program showed greater gains in reading 
skills than the comparison group students. Half of the 1990-91 and two thirds of the 
1991-92 Reading Recovery students were within the WCPSS average range for the 
Text Reading Level btest of the Diagnostic Survey at the end of first grade, 
compared to 15% of the non-program comparison group. 
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2. Results of E&R's analyses were somewhat more mixed, but generally positive, regarding 
Reading Recovery's long-term impact. Students who completed Reading Recovery, 
relative to comparison groups of similar students: 

• Had lower special education placement rates for the 1990-91 cohort, but not for the 
1991-92 cohort. 

• Had significantly lower Chapter 1 placement rates at the second grade level. (Data 
were available only for the 1990-91 cohort.) 

• Had significantly lower retention rates for the 1991-92 cohort, but not the 1990-91 
cohort. (Actual 1990-91 rates were lower, but few students were retained in either 
Reading Recovery or the comparison group.) 

• Were seen by second grade teachers as slightly better, but not significantly so, in 
terms of reading performance in class (1990-91 cohort). Fifty-five percent of Reading 
Recovery and 65% of non-program students were in the "low" reading group as of 
spring, 1992. It is important to note that these data do not provide a direct measure 
of student reading achievement, but rather represent teachers' perceptions of these 
students' reading performance in their classroom. 

3. About one fourth of those served by Reading Recovery each year did not receive a 
complete program. In most long-term follow-up analyses, these students did not fare 
better than non-program students. One notable exception was that; 1991-92 students 
receiving 30 lessons or less were referred to special education less often than non- 
program students or Reading Recovery completers. This finding may reflect the 
participation of many of these students in the Early Reading program while waiting for a 
Reading Recovery opening. 

ISSUES AND CONCERNS 

While the short-term Reading Recovery results were quite positive, and long-term results 
fairly positive, there are at least three areas for concern and possible rbllcw-up. The 
program itself seems quite promising, but regular classroom teachers may need more 
information to fully capitalize on the skills that Reading Recovery participation provides to 
students. One area which should be addressed is the lack of difference in special education 
placements in second grade for the 1991-92 Reading Recovery students compared to non- 
program comparison students. While Reading Recovery cannot have been expected to impact 
all of the reasons for special education referrals, at least some impact should have been 
detected. 
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A second concern is the second grade teachers' perceptions that many former Reading 
Recovery students were functioning below average in reading, and the lack of statistically 
significant difference relative to similar students who were not served. It appears second 
grade teachers (and perhaps some first grade teachers) may not fully recognize or capitalize 
on the skills and strategies students learn through Reading Recovery. Although these survey 
results do not provide a direct measure of actual student reading achievement, they suggest 
that regular classroom first and second grade teachers need more information and help in 
recognizing and building on students' Reading Recovery success. A future long-term study 
might involve readministering the Diagnostic Survey to Reading Recovery and non-program 
comparison students at the end of second grade, and comparing classroom teachers' 
perceptions of these students' reading ability with their performance on the Diagnostic 
Survey. 

Third, the percentage of Reading Recovery students who did not complete the program 
seemed somewhat high (about 25%), but this is comparable to a national Reading Recovery 
study. In WCPSS at least, this finding is partly the result of phasing students in as others 
are successfully exited. Program staff may want to consider whether it is wise to start 
students in the program if a full program, or at least a set minimum number of lessons, 
cannot be provided. Our study found less impact for these students, and a national study by 
Reading Recovery staff also found a lack of impact for an abbreviated Reading Recovery 
program (Groom, et al., 1992). 

Alternatives may be to provide more training to first and second grade classroom teachers 
later in the spring (and not phase in new students after a certain point in the year), or to find 
ways to allow students who start late to complete the program (either in the spring, summer, 
or fall of grade 2). 

The effectiveness of the Early Reading program, either alone or in combination with Reading 
Recovery, deserves further study. It may be that certain kinds of students could be identified 
that would benefit as much from the small-group Early Reading approach as the individual 
Reading Recovery lessons. (Both use Reading Recovery techniques.) Because the group 
approach is less expensive, this could help stretch tight resources and reach more students. 
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PROGRAM OVERVIEW 



BACKGROUND 

Traditional approaches to help children having difficulty learning to read have given children 
time to mature so that they could learn to read when they were developmental^ ready 
(Pinnell, DeFord, & Lyons, 1988). After first grade, students with insufficient reading skills 
were either retained in grade or were provided with compensatory education. Unfortunately, 
retention in grade and traditional compensatory education practices like pullout Chapter 1 
programs are frequently ineffective in bringing students up to grade level in reading (Slavin 
& Madden, 1989). The ineffectiveness of these approaches is probably a result of attempting 
to remediate students after they have already encountered problems learning to read and have 
likely developed negative attitudes about the process of reading because of repeated failure 
experiences (Slavin, 1991). 

Many researchers and practitioners have suggested that it is critical to prevent reading 
problems by providing intensive resources for first grade children who experience difficulty 
reading. One program that provides such services is Reading Recovery. This early 
intervention program attempts to reach first graders who are experiencing the greatest 
difficulty learning to read and provide them with services to help them catch up before they 
develop a pattern of school failure (Pinnell, et al., 1988). 

PROGRAM DESCRIPTION 

The goal of Reading Recovery is to help the lowest achieving 20% of first grade children 
become independent readers who use their own knowledge to solve problems they encounter 
during reading (Pinnell, Fried, 8c Estice, 1990). Teachers are trained in Reading Recovery 
techniques through an intensive, year-long inservice program in which they simultaneously 
work with children who need Reading Recovery services. During training, teachers enrich 
their knowledge of the reading process and diagnostic procedures used for accurate 
assessment of students' progress in learning to read. 

Reading Recovery teachers first use individual diagnostic procedures to identify students who 
will need help with reading. These teachers then develop individualized intervention 
procedures to help these students become independent readers and catch up with their 
average peers. Reading Recovery teachers provide daily 30-minute lessons until the child is 
capable of performing within the average range of his or her first grade classroom; once 
children exhibit this capability, they are discontinued from Reading Recovery services. 
Teachers make informed decisions to accelerate children's progress by closely assessing and 
monitoring their reading behavior as they move through the lessons. The instructional 
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approach combines the use of related reading and writing experiences, close interaction 
between teachers and children during lessons, and selection of appropriate reading materials. 

RESEARCH BASIS FOR BE ADPffl RECOVERY 

Reading Recovery teachers keep extensive records on children they serve The Reading 
Recovery program is part of the National Diffusion Network (NDN) and has been validated 
in many studies conducted nationally. The Reading Recovery program was developed and 
pilot tested in New Zealand in 1979 by researchers working with Dr. Mane Clay (Clay 
1985) Initial studies of the effectiveness of the program demonstrated that over 90% ot 
children who had a complete Reading Recovery program were able to reach the average 
levels of their classmates, and appeared to have developed an independent system of reading. 
The instrument used both to provide appropriate instruction and to evaluate the program is 
the Diagnostic Survey (Clay, 1985). Follow-up studies of these students indicated that these 
children continued to perform at average reading levels three years after services had been 
discontinued. 

In the 1991-92 school year, 74% of students served by Reading Recovery teachers nationally 
received complete programs. Eighty-four percent of students with a complete program were 
successfully discontinued from the program (Groom, et al., 1992). 

Recent research at Ohio State University (Pinnell, et al., 1988) compared two cohorts of first 
grade children served by Reading Recovery to matched control groups of children through 
the end of the third grade. At the end of first grade, Reading Recovery students who 
completed the program scored significantly higher than control students on the Stanford 
Reading Test. To follow children longitudinally, the Test Reading Level portion ot the 
Diagnostic Survey (Clay, 1985) was administered at ihe end of grades two and three. 
Reading Recovery children continued to significantly out-perform control children through 
the end of grade three, although effects diminished over time. Reading Recovery children 
were also less likely than control children to fail a grade up through the end of third grade. 
Research with a later cohort of Reading Recovery children demonstrated similar findings 
(DeFord, Pinnell, Lyons, & Young, 1987). 

Another Ohio State University study conducted in 1988 compared the effectiveness of the 
Reading Recovery program with three other methods used to correct reading difficulties. A 
total of 324 first graders who had tested as the lowest readers in their classes were assigned 
to one of four intervention programs or a control group representing traditional practice 
The first intervention program was Reading Recovery with trained teachers. The second 
program was led by teachers trained in an abbreviated Reading Recovery program. In the 
third intervention program, students interacted one-on-one with experienced reading teachers 
who did not use Reading Recovery techniques. In the fourth program, trained Reading 
Recovery teachers led group sessions instead of private lessons. Each program had its own 
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control group, which relied on skill drills and worksheets. Remedial instruction lasted 70 
days for each of the first four groups and the entire school year for the control classes. The 
study concluded that Reading Recovery completers out-performed children from an 
equivalent control group and the three other Intervention programs. In addition, Reading 
Recovery was the only intervention program that had lasting effects through the beginning of 
grade 2 (Groom, et al., 1992). 

These results indicate that the Reading Recovery model can have significant positive effects 
on the reading achievement of first grade children. Although the effects diminish somewhat 
over time, a positive impact continues even at the end of third grade in the absence of any 
special interventions in grades two and three. As a result of these research studies, the 
Reading Recovery program is being used in numerous sites across the nation as an early 
intervention program for students at risk of failure in reading. 

THE READING RECOVERY PROGRAM IN WCPSS 

WCPSS began implementing the Reading Recovery program in fall, 1990. The 1990-91 
school year was considered a pilot year because of the delayed beginning of the program; 
students did not begin receiving services until October, 1990. The firs* grade model used 
previouslv by Chapter 1 involved traditional pullout instruction with first grade students who 
had been retained. Twelve Chapter 1 teachers in ten qualifying Chapter 1 schools provided 
Reading Recovery services during the first year of implementation. Each teacher spent one 
half day working with individual first graders on Reading Recovery lessons and one half day 
teaching regular Chapter 1 groups. These Reading Recovery teachers also participated in a 
graduate-level course which included 33 training sessions from September, 1990, to June, 
1991. A total of 84 students received some level of Reading Recovery service during the 
1990-91 school year. 

During the second year of implementation (1991-92), the Reading Recovery program was 
expanded to include 20 Chapter 1 schools, and 12 additional Chapter 1 teachers received 
Reading Recovery training and provided Reading Recovery services. In addition, the Early 
Reading program was implemented for the first time to provide service to first grade students 
in small groups who were eligible for Reading Recovery but were on a waiting list to receive 
services. The Reading Recovery teacher taught both Reading Recovery lessons and Early 
Reading program groups. Early Reading program instruction was similar to Reading 
Recovery instruction, and centered around teaching students concepts about print and 
strategies that successful readers use during the reading process. Students received service in 
the Early Reading program until space became available for Reading Recovery instruction, or 
until they were able to effectively employ reading strategies and were reading at a 
satisfactory level in the regular classroom. One school in the program, Baucom, did not 
have an Early Reading program due to its non-Chapter 1 status. A total of 143 students 
received some level of Reading Recovery service during the 1991-92 school year. 
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For the third year of implementation (1992-93), the Reading Recovety program has been 
further expanded to include 32 Chapter 1 schools and four non- Chapter 1 schools, and 18 
additional teachers are receiving Reading Recovery training and providing Reading Recovery 
services. In addition, the Early Reading program is being provided in all Chapter 1 Reading 
Recovery schools in a manner similar to the model provided in 1991-92. 



>:\rdjrec\reEOrt\jd\gmw\3-5-93\Mirch 16, 1993 



4 i0 



93.09 



EVALUATION DESIGN 



SHORT-TERM FOLLOW-UP 

The Reading Recovery program is part of the National Diffusion Network (NDN), and 
therefore must be evaluated annually by program taff using the approved NDN research 
desif n This design involves systematically comparing the performance of students receiving 
a complete Reading Recovery program with a random sample of first grade students using 
the Clay Diagnostic Survey for the year of program service. Students are considered to have 
had a complete program if they have had at least 60 Reading Recovery lessons or were 
successfully discontinued from the program prior to having 60 lessons Students are 
"discontinued" (provided no further services) if they demonstrate reading skills on grade 
level based on the Clay Diagnostic Survey. The NDN model does not include any students 
in the analyses who do not complete 60 lessons and are not successfully discontinued. 

For comparison purposes, a random sample of students from each Reading Recovery site is 
selected for Diagnostic Survey testing to create a "site random band," or an average id of 
reading achievement levels at the end of first grade. The average band is then calculated by 
determining 0.5 standard deviations above and below the mean for alt students in the random 
sample. The average band is used to determine whether Reading Recovery students have 
moved into the average or above average range at the end of first grade. In addition, the 
group of Reading Recovery students is divided into a group of students who were 
Successfully discontinued from the program (discontinued group) and those who received at 
least 60 lessons but who were not discontinued from the program (non-discontinued group) ,o 
determine success rates for different types of Reading Recovery students. 

The three measures used within the Diagnostic Survey for the research design are Text 
Reading Level, Dictate, and Writing Vocabulary. The Writing Vocabulary subtest requires 
students to write all the wcrds they know within a ten-minute time limit. The Dictation 
subtest involves the teacher reading a sentence, and students writing the words as it is read. 
For the Text Reading Level subtest, students read text materials in graded levels ot 
difficulty, and their text reading level indicates the highest level of text that can be read at 
90% accuracy or above. 

WCPSS decided to extend the NDN evaluation model by including a comparison group of 

similar' low achievers who were not served by Reading Recovery. In 1990-91, a control 

group was established within the same Reading Recovery schools by serving every other 

elieible student. In 1991-92, a comparison group of students in non-Reading Recovery 

Sols was used who could have qualified for Reading Recovery if they had the program 

available to them The Clay Diagnostic Survey was administered to ail non- program 

common *uden,s to allow comparisons of .heir performance to that of Reading Recovery students. 
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LONG-TERM FOLLOW-UP 

RET ENTION. CHAPTER 1. A NT) SPECIAL EDUCATION STATUS 

The Chapter 1 evaluation specialist compared the performance of Reading Recovery and non- 
program comparison students to determine the long-term impact of the Reading Recovery 
program. For the 1990-91 cohort of students, the mainframe student database provided 
information for the following school year (1991-92) to determine how many students in each 
group had been retained or had been promoted to second grade. In addition, the database 
indicated whether these students were receiving either Chapter 1 services or some type of 
special education program services as second graders. This analysis is being repeated in 
1992-93 when these children are in third grade. The 1991-92 cohort of students is also being 
followed in a similar fashion to determine if they are receiving special education or were 
retained as first graders following the 1991-92 school year. 

in addition to analyzing results for the Reading Recovery students with a complete program 
(called Reading Recovery program students) and the non-program comparison students, we 
also examined the retention, special education, and Chapter 1 status of students who received 
Reading Recovery services but did not complete the program. These students were divided 
into those who left the program with 30 lessons or more (at least 15 hours of instruction) or 
less than 30 lessons. Students "left" the program early for any of the following reasons: 
1) transferred with less than 60 lessons, 2) were in the program at the end of the year with 
less than 60 lessons, 3) withdrew to special programs, or 4) withdrew with either more or 
less than 60 lessons for some other reason. Definitions of all groups included in these 
analyses are shown on the next page. 

SURVEY INFORMATION 

Norm -referenced tests are not administered at the second grade level in WCPSS except to 
students who are considered potentially eligible for Chapter 1 or gifted services. Therefore, 
to obtain a measure of reading performance, second-grade classroom teachers who taught the 
1990-91 cohort of Reading Recovery completers and non-program comparison group students 
completed a survey in April, 1992, concerning these students' performance as readers within 
their classroom during the 1991-92 school year. Questions focused on the students' level of 
performance relative to others in the class (e.g., middle reading group versus low reading 
group) as well as how they performed in different areas of reading (e.g., vocabulary versus 
comprehension). Although the survey information does not provide a direct measure of 
student reading performance, it does provide information on teachers' perceptions of 
students' ability (which can impact student performance). 
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GROUP DEFINITIONS 



Throughout this report, the following terms are used refer to Reading Recovery and 
comparison groups of students. 

READING RECOVERY STUDENTS 

Completers : Students who received a complete program by having at least 60 Reading 
Recovery lessons and/or were successfully discontinued from the program. Sradents are 
'discontinued' from program services when they successfully show on-grade level 
reading skills based on the Clay Diagnostic Survey. Other students who complete 60 
lessons but are unable to successfully read on a level equal to or better than average first 
graders are called non-discontinued students. For short-term analyses involving results 
from the site report, these students are described simply as Reading Recovery students, 
but for long-term analyses these students will be referred to as Reading Recovery 
"completers." 

Non-Completers : Students served by Reading Recovery who did not receive a complete 
program because they 1) transferred with less than 60 lessons, 2) were in the program at 
the end of the year with less than 60 lessons, 3) withdrew to special education programs, 
or 4) withdrew with either more or less than 60 lessons for some other reason. These 
students were divided into two groups: those who received less than 30 Reading 
Recovery lessons, and those who received more than 30 Reading Recovery lessons. 

NON-PROGRAM COMPARISON STUDENTS 

Random Band : Randomly selected first graders in Reading Recovery schools used as a 
comparison group in NDN studies. These students are given the Clay Diagnostic Survey 
in the spring to determine the "average band" of first grade reading performance. 

Non-Program Comparison : Students in the control group in 1990-91 or the comparison 
group in 1991-92. The control group in 1990-91 included students in the same school 
who qualified and were not served on an alternating basis to check program effects. The 
1991-92 students were in schools comparable to Reading Recovery schools in which the 
Reading Recovery program was not available. 
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Results from these follow-up analyses and a brief description of the results from the NDN 
required evaluation are provided in the next sections. These sections will cover the 
following: 1) highlights from the program staffs report from the one-year NDN evaluation; 
2) the long-term impact of Reading Recovery on 1990-91 students; and 3) a similar 
description of results for the 1991-92 cohort of students. 
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EVALUATION RESULTS: 1990-91 COHORT 



SUMMARY OF FINDINGS 



he major findings from the evaluation of the 1990-91 cohort were: 

• Diagnostic Survey results indicated that Reading Recovery completers were 
performing at higher levels than non-program comparison students on all subtests at 
the end of first grade, and more were meeting or exceeding the average band for the 
first grade. 

• Reading Recovery completers were somewhat less likely to be retained at the end of 
first grade than non-program comparison students, but the small number of retainees 
limits the conclusions which can be drawn. Both groups of non-completers (less 
than and more than 30 lessons) were more likely to be retained both years than 
either Reading Recovery completers or non-program comparison students, but the 
difference was not statistically significant. 

• Reading Recovery completers were less likely than non-program comparison 
students to receive special education services in subsequent years: 6% of Reading 
Recovery completers and 12% of non-program comparison students received special 
education services in 1991-92 or 1992-93. In 1991-92, non-completers with less 
than 30 lessons were more likely than both Reading Recovery completers and non- 
program comparison students to receive special education services, but only the 
comparison with Reading Recovery completers was significant. In 1992-93, non- 
completers with less than 30 lessons were significantly more likely than completers 
and non-program comparison students to receive special education services. 

• Reading Recovery completers were significantly less likely than non-program 
comparison students to receive Chapter 1 service in 1991-92: 33% of completers 
participated in Chapter 1, while 51% of non-program comparison students 
participated in Chapter 1. Both groups of non-completers were similar to Reading 
Recovery completers in terms of Chapter 1 participation rates. 

• Second grade teacher survey results were somewhat equivocal. No consistent 
differences emerged between Reading Recovery and non-program comparison 
groups in terms of teacher reports of reading performance. However, there was a 
consistent difference favoring discontinued over non-discont;nued Reading Recovery 
students on all items addressed in the survey. 
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SHORT-TERM FOLLOW-UP 

The WCPSS site report (Hundley, 1991) was prepared by Reading Recovery program staff 
and focused on the discontinuation rate of Reading Recovery students and on Diagnostic 
Survey results in spring, 1991, for Reading Recovery and non-program comparison group 
students. Reading Recovery students were considered to be any students who had at lea 60 
lessons and/or who were discontinued from the program. Non-completers were not included 
in analyses for the WCPSS site report. Key findings from the WCPSS Reading Recovery 
1990-91 site report were: 

• Seventy-seven percent of students who had a complete Reading Recovery program 
were successfully discontinued from the program. 

• Scores for Reading Recovery students were lower than those for the non-program 
comparison group of students in the fall (pretest), but higher in the spring (posttest) 
for both the Writing Vocabulary and Dictation subtests of the Diagnostic Survey. 

• Reading Recovery completers were reading on a level approximately equivalent to the 
end of the first grade reader; non-program comparison group students read at a level 
comparable to the third pre-primer based on the Text Reading subtest of the 
Diagnostic Survey. 

• On the average, Reading Recovery completers were able to read within the average 
band for WCPSS first graders, while non-program students were not The 
percentage of Reading Recovery students who scored equal to or higher than the site 
average band on the posttest was 80% for Writing Vocabulary, 61% for Dictation, 
and 49% for the Text Reading subtest. The percentages for non-program comparison 
students was much lower (45% for Writing Vocabulary, 35% for Dictation, and 15% 
for the Text Reading subtest). 

• When the group of Reading Recovery completers was divided into discontinued and 
non-discontinued students and the results for the two groups analyzed separately, 
results indicated that although non-discontinued students were able to achieve growth 
on each of the subtests of the Diagnostic Survey, their progress was not as 
accelerated as that of the discontinued students. 

• All principals at Reading Recovery schools and 98% of parents of Reading Recovery 
students indicated on a survey that the Reading Recovery program was a good or 
very good program. In addition, 94% of classroom teachers at Reading Recovery 
schools responded that Reading Recovery was an effective program for reducing 
reading failure. 
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Overall, results for the first year of implementation of the Reading Recovery program were 
quite positive. However, program staff expressed concern regarding the number of non- 
discontinued program students who made gains from fall to spring, but who failed to reach 
reading levels that correlated with average first grade achievement. Reasons for the failure 
of these students to make sufficient progress were cited as: 1) teachers simultaneously 
providing instruction while learning how to implement Reading Recovery, 2) poor student 
attendance, and 3) need for long-term special programs assistance. 

m Based on the success of the Reading Recovery program in 1990-91, the decision was made to 
train additional Chapter 1 teachers and expand the program to additional schools. 
Recommendations were made to monitor more closely the progress of students not showing 
signs of making significant progress, It was planned that the teacher leader would be 
consulted when a Reading Recovery student was not beginning to progress by thr ^Oth 
lesson. If necessary, a plan for improving instruction or referral to the school's assistance 
team would then be developed in cooperation with the classroom, teacher. In addition, 
increased communication between the Reading Recovery teachers and other school staff was 
encouraged to deal with problems of poor attendance. 

LONG-TERM FOLLOW-UP 

SERVICE PROVIDED 

The Reading Recovery site report emphasizes program outcomes for students receiving a 
complete program (those with at least 60 lessons and/or discontinued). However, because 
resources are also spent on students who only receive a partial Reading Recovery program 
(less than 60 lessons without discontinuation), E&R staff included these students in analyses. 
One question of interest was the completion rate for the program, i.e., the number and 
percent of students who left the program before completing Reading Recovery services and 
the amount of service received. The other major question was whether the Reading 
Recovery program had any long-term impact on these "non-completers" who received only a 
partial program. 

Figure 1 shows that 61 (or 73%) of the 84 students served in 1990-91 received a complete 
program; 23 (or 27%) did not. Of the 23 "non-completers," most (74%) received 30 lessons 
(15 hours) of service or less. The reasons students left the program were varied. Of the 
non-completers: 

1) Sixteen (70%) had fewer than 60 lessons at the end of the year; 

2) Four transferred to other schools with less than 60 lessons; 

3) Two withdrew to special education programs; and 

4) One withdrew for another reason. 
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While a logical assumption for many supplementary programs would be that students 
receiving less service would be less likely to benefit, delivery methods in Reading Recovery 
make this assumption invalid. In fact, the opposite could also occur, because: 1) students in 
less need are served as others successfully leave the program, and therefore would be more 
likely to have fewer lessons, and 2) students who receive more lessons are likely to be those 
with the greatest need (others are discontinued sooner). 

OVERVIEW OF ANALYSES 

In order to obtain a complete picture of the impact of the Reading Recovery program, data 
analyses for retention, special education, and Chapter 1 status focused on two types of group 
comparisons. One set of analyses focused on comparing all Reading Recovery completers 
(successfully iiscontinued and non-discontinued students with at least 60 lessons) with non- 
program comparison students. In addition, differences among types of Reading Recovery 
students (discontinued versus non-discontinued) will be reported when group differences are 
large, and summary tables for these analyses are available upon request. The second set of 
analyses focussed on results for students who did not receive a complete Reading Recovery 
program (non-completers who received less than 60 lessons). 

When statistically significant differences occurred between groups, Chi Square or Fisher's 
Exact (for small cell sizes) statistics will be reported. Figure 1 below illustrates the 
breakdown of the sample used for the analyses. 



Figure 1 

1990-91 All Reading Recovery Students Served 




30 Lessons or Less 



Non-Completers 
N = 23 (27%) 



Completers 
N = 61 (73%) 



31-60 Lessons 
N=4 



Unknown' 
N = 2 



•Left WCPSS 



Discontinued 
N = 47 
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RETENTION 

The retention policy in WCPSS indicates that decisions are made based on academic 
performance, developmental considerations, physical/social maturity, attitudes, and previous 
retention status (only one retention is allowed in grades K-5). Ti»us, a combination of 
objective and subjective criteria are used. In 1990-91, 1.9% of first graders and 0.87% of 
second graders were retained in WCPSS. 

Figure 2 provides the frequency data on 1990-91 Reading Recovery and non-program 
comparison student retention rates following the 1990-91 and 1991-92 school years. Results 
from these analyses indicated: 

• More non-program comparison group students than Reading Recovery completers 
were retained after 1990-91, but retention rates for the two groups were approximately 
the same after 1991-92. 

• A greater percentage of both groups of non-completers were retained than Reading 
Recovery completers or non-program comparison students. 

Retention rates for each group were not significantly different for 1990-91 retention, possibly 
because of the small number of children being retained in each group. Therefore, no 
.definitive conclusions can be drawn as to whether the Reading Recovery program had an 
impact on student retention. 



ERLC 
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Figure 2. Retention Status for 19<>0-91 Reading Recovery Completers, 
Non-Completers, and Non-Program Comparison Students 
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*Represents one discontinued and two non-discontinued students. 



SPECIAL EDUCATION STATUS 

Figure 3 provides the numbers and percentages of Reading Recovery and non-program 
comparison group students who received or are receiving special education services during 
the 1991-92 and 1992-93 school years. Some of the students received or are receiving 
special education services in both 1991-92 and 1992-93; therefore, an additional column has 
been added to the table to represent Only the total number of individual students receiving 
special education in either 1991-92 or 1992-93. 
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Figure 3. Special Education Status for 1990-91 Reading Recovery 
Completers, Non-Completers, and Non-Program Comparison Students 
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Special Education 
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Non-Program 
Comparison 
Students 
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10 (13%) 


65 (87%) 


5 (7%) 


70 (93%) 


9 (12%) 



♦Represents two discontinued and one non-discontinued students. 

These results demonstrate that students in the non-program comparison group were more 
likely to receive special education services as second graders than were Reading Recovery 
students- overall, 12% of non-program comparison students received services, while only 
6% of Reading Recovery students received services. The numbers of students receiving 
special education services declined in 1992-93 for both groups (when students were in third 
grade). 

Non-completers with 30 Reading Recovery lessons or less were significantly more likely to 
receive special education services than Reading Recovery students in both 1991-92 (Fisher s 
Exact Test, p = .01) and 1992-93 (Fisher's Exact Test, p = .003), but were not more likely 
than non-program comparison students to receive special education services in either school 
year. 
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When the type of special education placement was examined, it was found that one Reading 
Recovery discontinued student received learning disability services tt td one received speech 
services, while the non-discontinued student received speech services. The student who was 
the non-completer with more than 30 lessons was receiving learning disability services, while 
of the non-completer group, two were receiving learning disability services, two were 
receiving speech services, and one was receiving educationally mentally handicapped 
services. 

CHAPTER 1 STATUS 

Students are eligible to receive Chapter 1 services if they score at or below the 49th 
percentile on the reading portion of the California Achievement Test (CAT). Figure 4 
provides the numbers and percentages of Reading Recovery and non-program comparison 
students who received Chapter 1 services during 1991-92. 



Figure 4. Chapter 1 Status for All Reading Recovery Students 
and Discontinued, Non-Discontinued, and Non-Program 
Comparison Students 





Participated in Chapter 1 in 
1991-92 


Reading Recovery Students 
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Completers 
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♦Represents 10 discontinued and 7 non-discontinued students. 



In terms of Chapter 1 service in 1991-92, these data indicate that: 

• Students with a complete Reading Recovery program were significantly less likely 
than the non-program comparison group to be served (Chi Square, X=3.71, 
p =0.054); 
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• Students who did not complete Reading Recovery had similar Chapter 1 service 
rates to Reading Recovery completers (which were tower than non-program 
comparison students but not significantly so); 

• Within the Reading Recovery completers, those successfully discontinued were 
significantly (X=4.41, p=0.036) less likely to receive Chapter 1 than those not 
discontinued (see Figure 5); and 

• Reading Recovery non-discontinued students had rates of service similar to non- 
program comparison students (see Figuic 5). 



Figure 5. Percentage of Reading Recovery Completers and 
Non-Program Comparison Students Participating in Chapter 1 

in 1991-92 




Rdg. Rtc. Rdg. Rsc. Rdg. Rec. Non-Program 

Compl«ttrt Discontinued Non- Comparison 

Discontinued 



SECOND GRADE TEACHER SURVEYS 

Second grade teachers of Reading Recover)' and non-program comparison students were sent 
surveys in April, 1992, for each student that they currently had in their classrooms who was 
in the evaluation sample. (Reading Recovery non-completers were not included in the 
sample for teacher surveys.) They were not told the group status (Reading Recovery or non- 
program comparison) of any of these students. A copy of the survey can be found in the 
Appendix. 

Reading Levels 

Figure 6 provides the survey results for an item which asked second grade teachers to 
indicate students' beginning reading levels based on the information they received about the 
students at the beginning of the school year. 
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Figure 6. Teacher Survey Responses: Second 
Graders' Beginning Reading Level (Fail, 1991) 
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Teachers indicated that, based on the information from first grade they received at the 
beginning of the school year, Reading Recovery students were more likely than non-program 
comparison students to enter second grade on grade level in terms of reading skill, and this 
difference was significant (X = 4.095, p = 0.043). Subsequent analyses indicated that 
significantly more Reading Recovery discontinued students than non-program comparison 
students were likely to have beginning reading levels that were on grade level (X = 7.317, p 
= 0.007). A significant difference favoring Reading Recovery discontinued students over 
non-discontinued students also existed (X = 4.479, p = 0.034), but no significant difference 
was found between non-discontinued and non-program comparison students in terms of 
beginning reading level. 

It should be noted, however, that only 26% of the Reading Recovery students were 
considered "on grade level." This is somewhat puzzling in that 49% had text reading within 
the average band based on the Clay Diagnostic Survey results for Reading Recovery and 
random first graders the previous spring. It may be that first grade teachers conveyed this 
information incorrectly or that second grade teachers blended information received with their 
recollections of students' fall performance. 

Teachers were also asked to report on their students' current reading level (as of April, 
1992), and these results are provided in Figure 7. 
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Figure 7, Teacher Survey Responses: 
Students' Current Reading Level 





Below Grade 
Level 


On Grade Level 


Above Grade 
Level 


Reading 
Recovery 


43% 


53% 


5% 


Non-Program 
Comparison 


47% 


51% 


2% 



These results suggest some improvement in teachers' perceptions of student performance for 
both groups and very little difference in terms of current reading levels between Reading 
Recovery and non-progr? n comparison students: 53% of Reading Recovery and 51% of 
non-program compariso . .,jdents were considered to be reading on grade level. When 
Reading Recovery discontinued and non-discontinued students were examined separately, 
discontinued students were more likely to be reading on grade level (59%) than non- 
discontinued students (25%), The definition of "on grade level" is problematic and may have 
varied across respondents. Some may have meant students were in any second grade reader 
(whether behind the class average or not), while others may have meant students had caught 
up to the average for the class. 

Instructional Reading Groups 

Teachers were also asked to indicate what reading group students were participating in when ' 
grouping practices were used. Figure 8 reports the instructional reading group participation 
for Reading Recovery and non-program comparison students. 



Figure 8. Teacher Survey Responses: 
Student Participation in Reading Groups 
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This figure shows that teachers reported that non-program comparison students were 
somewhat (10%) less likely than Reading Recovery students to participate in an average and 
high reading group, and more likely to partic ; pate in a low reading group than Reading 
Recovery students. However, these differences were not significant. 

When survey responses were examined for Reading Recovery discontinued and non- 
discontinued students separately, results showed that 47%, 41%, and 12% of Reading 
Recovery discontinued students participated in the low, average, and high reading groups, 
respectively. The results fnr Reading Recovery non-discontinued students indicated that 88% 
of these studenr were in low reading groups, and only 13% were in average reading groups; 
there were no non-discontinued students participating in a high reading group. 

Enjoyment of Reading 

Teachers responded to an item which asked them to rate how much they believed students 
appeared to enjoy reading (see Figure 9). 



Figure 9. Teacher Survey Responses: 
Student Enjoyment of Reading 
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These results show that teachers believed that Reading Recovery students were more likely to 
enjoy reading an average amount (40%) than non-program comparison students (29%), and 
non-program comparison students were more likely to be viewed as enjoying reading less 
than average (41%) than Reading Recovery students (33%). None of these group differences 
were statistically significant. 

Reading Skill 

Teachers were asked to rate students' reading skill level in three areas of reading on a scale 
from one to five; this scale was then collapsed for analysis purposes to derive categories of 
low, average, and high reading skill levels. Figures 10, 11, and 12 provide the results for 
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teachers' responses to students' skill in word analysis, vocabulary and comprehension, 
respectively. 

Word Analysis , Figure 10 demonstrates that just over half (51%) of the Reading Recovery 
students were rated as having a low level of skill in word analysis, and 56% of non-program 
comparison students were given this rating. Somewhat more non-program comparison 
students (32%) than Reading Recovery students (26%) were rated as having an average level 
of skill in word analysis, while 23% of Reading Recovery students were rated as having a 
high level of skill in this area as compared to 12% of non-program comparison students. 

Figure 10. Reading Skill: Word Analysis 
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Separate results for Reading Recovery discontinued and non-discontinued students indicated 
that 45% of discontinued students were rated as having low levels of skill and 27% were 
rated as having average and high levels of skill in word analysis. For non-discontinued 
students, 70% were rated as having low levels of skill, 20% were rated as having average 
levels of skill, and 10% were rated as having high levels of skill in word analysis. 

Vocabulary , Figure 11 shows teacher responses to rating students' skill level in vocabulary. 
Fewer students were rated as having a low level of skill in vocabulary than in word analysis, 
with 44% of Reading Recovery and 40% of non-program comparison students receiving this 
rating. More non-program comparison students were rated as having an average level of 
skill in vocabulary (38%) than Reading Recovery students (30%), while more Reading 
Recovery students were rated as having a high level of skill (26%) than non-program 
comparison students (22%). 
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Figure 11. Reading Skill: Vocabulary 
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When results were examined separately for Reading Recovery discontinued and non- 
discontinued students, it was found that 33% of discontinued students were rated as having a 
low level of skill, 39% were rated as having an average level of skill, and 27% were rated as 
having a high level of skill in vocabulary. A total of 80% of non-discontinued students were 
rated as having a low level of skill and 20% were rated as having a high level of skill; no 
non-discontinued students were rated as having an average level of skill in vocabulary. 

Comprehension . Figure 12 provides teacher responses when asked to rate students' skill 
level in comprehension. Reading Recovery students' skill level in comprehension was rated 
somewhat similarly to their word analysis skills: 51% were rated as having a low level of 
skill, 23% were rated as having an average level of skill, and 26% were rated as having a 
high level of skill in comprehension. Fewer students in the non-program comparison group 
than in the Reading Recovery group were rated as having a low level of skill in 
comprehension (41%), and more students in the non-program comparison group were rated 
as having an average level of skill (37%) than Reading Recovery students (23%) in 
comprehension. 
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Figurt 12. Reading Skill: Comprehension 
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Reading Recovery discontinued students again fared better than non-discontinued students in 
terns of teachers' rating of their skill in comprehension. A total of 42% of discontinued 
students were rated as having a low level of skill, 30% were rated as having an average level 
of skill, and 27% were rated as hav : :ig a high level of skill in comprehension. Results 
indicated that 80% of non-discontinued students were rated as having a low level of skill and 
20% were rated as having a high level of skill in comprehension, and no non-discontinued 
students were rated as having an average level of skill in comprehension. 
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EVALUATION RESULTS: 1991-92 COHORT 



SUMMARY OF FINDINGS 

Z major findings from the evaluation of the 1991-92 cohort were: 

. Diagnostic Survey results ^^^^J^Stn^ 
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. Reading Recovery completers were ^g« less 
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SHORT-TERM FOLLOW-UP 
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the first grade reader, while non-program comparison students read at a level 
comparable to the third pre-primer. 

• The percentage of Reading Recovery completers who scored equal to or exceeded the 
site average band on the posttest was 78% for Writing Vocabulary, 83% for 
Dictation, and 63% for the Text Reading Subtest. The percentages for non-program 
comparison students were somewhat lower for Writing Vocabulary (55%) and 
Dictation (61%), but much lower for Text Reading (24%) than for Reading Recovery 
students. 

• When the group of Reading Recovery completers was divided into discontinued and 
non-discontinued students and the results for the two groups analyzed separately, 
results indicated that although non-discontinued students were able to achieve growth 
on each of the subtests of the Diagnostic Survey, their progress was not as 
accelerated as that of the discontinued students. 

• All principals at Reading Recovery schools and 99% of parents of Reading Recovery 
students reported on a survey that Reading Recovery was a good or very good 
program. In addition, 94% of classroom teachers at Reading Recovery schools 
agreed that Reading Recovery was an effective program for reducing reading failure; 
89% believed that Reading Recovery students were likely to continue to perform well 
in reading after the school year. 

Results for the second year of implementation of the Reading Recovery program were again 
positive. Although the discontinuation rate declined slightly from 1990-91, students in 
Reading Recovery in 1991-92 performed somewhat better than students in 1990-91 on the 
Diagnostic Survey. A greater percentage of students in 1991-92 scored within the average 
band and larger gains were seen from pretest to posttest as compared with 1990-91 students. 
The Reading Recovery program was again expanded for 1992-93 to train more teachers and 
serve more schools. 

Program staff again, however, expressed concern regarding the number of non-discontinued 
students who were not able to make the accelerated gains necessary to allow them to be 
reading on a first grade level. One reason cited for the failure of these students to make 
sufficient progress was that in-training teachers are simultaneously providing instruction 
while learning how to implement the program. Some students may have required an 
experienced Reading Recovery teacher who was better able to diagnose a problem more 
quickly and teach to that focus in order to accelerate student progress. The fact that trained 
Reading Recovery teachers discontinued a higher percentage of students than in-training 
teachers seems to support this possibility. 
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Poor program attendance was again cited as another possible reason for failure of some 
students to make sufficient gains; in fact, some of the non-discontinued students only 
attended Reading Recovery lessons three times weekly, and this may have inhibited 
accelerated growth. In addition, qualification for special programs services is another 
potential reason for the failure of non-discontinued students to make sufficient gains; 46% of 
these students tested at levels correlating with the first pre-primer in spring testing, and past 
program experience has indicated that these students are likely to qualify for EMH services 
the following year. 

LONG-TERM FOLLOW-UP 
SERVICE PROVIDED 

Figure 13 describes the level of service provided and completion status for all students 
involved in Reading Recovery in 1991-92. The percentage of students who received a 
complete Reading Recovery program (72%) was quite similar to 1990-91. However, among 
the non-completers, a larger percentage of students received 30 lessons or more in 1991-92 
than in 1990-91 (about half received 30 lessons or more in 1991-92 compared to about one 
fifth in 1990-91). 



The addition of the Early Reading program qualitatively changed the nature of service for 
non-completers in 1991-92. Students on the Reading Recovery waiting list (those with less 



Figure 13 

1991-92 All Reading Recovery Students Served 
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with less need) were served in a group setting until an opening became available. Thus, 
most students in the 30 lessons or less category probably had this additional service. 
Because they had less need initially and had a combination of Early Reading group service 
and Reading Recovery individual service, it was not assumed that they would do less well in 
terms of long-term outcomes than either the Reading Recovery completers or those who 
received 31-60 lessons. It was considered possible that the 1991-92 non-completers would 
show somewhat better long-term results than the 1990-91 non-completers because of this 
additional service. 

Students who did not complete the program fell into the following categories: 

• Twenty-two (54%) were still in the program with less than 60 lessons at the end of 
the year; 

• Nine (22%) had withdrawn to special education (eight were classified as learning 
disabled); 

• Five (12%) transferred from a Reading Recovery school before receiving 60 
lessons; and 

• Six (15%) withdrew for other reasons (two with more than 60 lessons). 
OVERVIEW OF ANALYSES 

Progress of completers, non-completers, and non-program comparison students was tracked 
to determine one-year rates for retention and special education. Chapter 1 status for 1992-93 
could not be checked because the database was not finalized. Data analyses focused on two 
types of group comparisons. One set compared all Reading Recovery completers 
(discontinued and non discontinued students with at least 60 lessons) with non-program 
comparison students. In addition, differences among types of Reading Recovery students 
(discontinued versus non-discontinued) were examined, and will be reported '"hen group 
differences are large (summary tables for these analyses are available upon request). The 
second set of analyses focused on comparing non-completers with completers and non- 
program comparison students. When statistically significant differences occurred between 
groups, Chi Square statistics will be reported. 

RETENTION 

Figure 14 shows the frequency data on 1991-92 Reading Recovery and non-program 
comparison student retention rates following the 1991-92 school year. These results 
demonstrate that: 

• Non-program comparison students were significantly more likely than Reading 
Recovery completers to be retained at the end of first grade (X = 12.67, p = 
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While 21% of non-program comparison students were retained, only 4% of Reading 
Recovery students were retained following the first grade. 

• The pattern of retention for non-completers more closely resembled the non-program 
comparison group of students: both groups of non-completers were more likely to bi 
retained than Reading Recovery completers (more than 30 lessons: Fishers' Exact 
Test, p=.07; less than 30 lessons: Fishers' Exact p = .02). 



Figure 14. Retention Status for 1991-92 Reading Recovery and 
Non-Program Comparison Students 





Retention Status After 1991-92 


Yes 


No 


Reading Recovery Students 












Non-complew* (3 1 to 60 te*cmi) 


. 3 (16%) 






4(19%) 




Non-Program Comparison 
Students (N=87) 


18 (21%) 


69 (79%) 



•Represents two discontinued and two non-discontinued students. 



SPECIAL EDUCATION STATUS 

Figure 15 provides the numbers and percentages of Reading Recovery and non-program 
comparison group students who were receiving special education services dunng 1992-93. 
Of the group of Reading Recovery discontinued students, six were receiving learning 
disability services, five were receiving speech services, and one was classified as other health 
impaired. Of those who were non-discontinued, six were receiving learning disability 
services two were receiving speech services, and one was receiving emotionally handicapped 
services. All of the non-completers in both groups were receiving learning disability services 
except one student, who was classified as other health impaired. 



34 ! 
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Figure 15. Special Education Status For 1991-92 Reading Recovery and 
Non-Program Comparison Students 





Special Education Status in 1992-93 


Yes 


No 


Reading Recovery Students 






Completers (N* 101} 








72- (22%) 




79 (78%) 


(mm 




:::S:::::?:::Srl;:wS:SS 

- liil 


11(58%) 


Noa-coaapleter* 00 or fewer |||§§ 
kmm) (N-21) 




,-,;,;,;,;r,rn. 

1 V : 1 
II ill 
• • 


ft,;;.;,,,;., „. — » 
19 (90%) 


Non-Program Comparison 
Students (N=87) 


22 (25%) 




65 (74%) 



*Represents 13 discontinued and 9 non-discontinued students. 



These results demonstrate that: 

• Reading Recovery completers were participating in special education services at 
approximately the some rate as the non-program comparison students. When results 
were examined separately for Reading Recovery discontinued and non-discontinued 
students, 17% of discontinued, and 38% of non-discontinued students were 
participating in special education programs for the 1992-93 school year. 

• Non-completers with more than 30 lessons were significantly more likely to be 
receiving special education services than Reading Recovery completers (Fishers' 
Exact, p=.06). However, non-completers with less than 30 lessons were less likely 
than Reading Recovery or non-program comparison students to be receiving special 
education services, but the difference was not statistically significant. This finding 
may be due to student participation in the Early Reading program. Most of the 
students with less than 30 lessons entered the program late after receiving Early 
Reading program services (see description on page 3), and may have benefitted from 
this instruction. On the other hand, non-completers with more than 30 lessons 
primarily consisted of students who exited Reading Recovery to special education 
programs. 
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SURVEY OF STUDENT READING ABILITIES 

Please complete one survey for this child who is currently in your class: 



If you do not currently have this student in your class, please simply return the blank survey. 
Feel free to elaborate on any item; your comments are welcome and desired for each item. 
Return completed surveys to Jan Donley, Department of Evaluation & Research. We appreciate 
your cooperation! 

Circle tlie response which most closely matches your perception: 

1. According to the reading information you received on this child at the beginning of the 
school year, what was this child's reading level when he/she entered your classroom? 

Below Grade On Grade Above Grade 

Level Level Levei 

2. Currently, is this child reading: 

Below Grade On Grade Above Grade 

Level Level Level 

3. In what type of reading program is this child participating? 

Basal Whole Language Combo Other 

(Basal & (comment) 

Whole 

Language) 



4. When you group your students for reading instruction, what reading group is this child 
in? 

Low Middle Average Middle High 

Low High 



(over) 
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5. Describe this child's reading strengths. 



6. ' Describe any areas this child needs improvement in as a reader. 



7. How much do you believe this child enjoys reading? 

Not at A little An average M ^* a " ^ch 

all amount average mucn 

8. Rate on a scale from 1 to 5, this child's skill level in the following areas related to 
reading behavior: 

Low H >g h 

Word analysis 1 2 3 4 5 

Vocabulary 1 2 3 4 5 

Comprehension 1 2 3 4 5 
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